Timely Feedback in Unstructured Cybersecurity Exercises

摘要

Cyber defence exercises are intensive, hands-on learning events for teams ofprofessionals who gain or develop their skills to successfully prevent andrespond to cyber attacks. The exercises mimic the real-life, routine operationof an organization which is being attacked by an unknown offender. Teams oflearners receive very limited immediate feedback from the instructors duringthe exercise; they can usually see only a scoreboard showing the aggregatedgain or loss of points for particular tasks. An in-depth analysis of learners'actions requires considerable human effort, which results in days or weeks ofdelay. The intensive experience is thus not followed by proper feedbackfacilitating actual learning, and this diminishes the effect of the exercise. In this initial work, we investigate how to provide valuable feedback tolearners right after the exercise without any unnecessary delay. Based on thescoring system of a cyber defence exercise, we have developed a new feedbacktool that presents an interactive, personalized timeline of exercise events. Wedeployed this tool during an international exercise, where we monitoredparticipants' interactions and gathered their reflections. The results showthat learners did use the new tool and rated it positively. Since this newfeature is not bound to a particular defence exercise, it can be applied to allexercises that employ scoring based on the evaluation of individual exerciseobjectives. As a result, it enables the learner to immediately reflect on theexperience gained.